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ABSTRACT 

The effects of level of adjurct guesticr (factual, 
application) and type of fetJback (no fe6..back, correct answer 
feedback, self -correction feedback) on learring ccrcepts from prose 
were examined in a study of 135 volunteer undergraduates at Iowa 
State University. Adjunct applicaticn guesticns prcdtce tetter 
pexforaance on subsequent new applicaticn items. This result 
supported the conclusion that practicing apflicaticn fchile -tudying 
facilitates later use cf the concept. Feedback influenced performance 
on the question asked during instruction, tut net cn re^i factual or 
application questions. These results suggested that college students 
need training in using feedback to learr concepts. (Authci) 
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Abstract 

The effects of level of adjunct question (factual, application) and type of feed- 
back (no feedback, correct answer feedback, self-correction feedback) on learning 
concepts from prose were examined. Adjunct application questions produced better 
performance on subsequent new application items. This result supported the con- 
clusion that practicing application while studying facilities later use of the 
concept. Feedback influenced performance on the questions asked during instruction, 
l.t nc. on new factual or application questiors. These results suggested that 
college students need training in using feedback to learn concepts. 
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Level of Adjunct Question, Type of Feedback 
and Learning Concepts from Prose 

Educators generally believe that asking higher level questions of students 
during instruction facilitates learning, retention, and the development of thinking 
skills. A number of reviewers have indicated that empirical support for this be- 
lief is weak (Andre, Note 1; Anderson and Biddle, 1975; Dunkin and Biddle, 1974) • 
Moreover, little is known about the conditions under which question level influences 
instruction. Some studies (Watts & Anderson, 1971; Dapra & Felker, 1975) have ap- 
pare..-iy demonstrated beneficial effects for higher level questions; while other 
studies (Anderson & Biddle, 1975; Andre, 1976) have found superior performance for 
low level questions. Andre and Sola (1976) argued that question level might inter- 
act with feedback and the opportunity to restudy the instructional material. Their 
study demonstrated that subjects who received higher level test items and who also 
had the opportunity to restudy did better on a final test containing new higher level 
items. Subjects who received high level items but did not have the opportunity to 
restudy did no b^^tter than subjects who received low level test items. Andre and 
Sola (1976) argued that test item level influenced learning most positively when 
combined with feedback and the opportunity to restudy. 

Ifliile the results of the Andre and Sola (1976) study are suggestive, certain 
facts raise questions about the generalizability of their findings. The materials 
consisted of an arbitrary list of sentences and subjects merely had to memorize the 
subject nouns. Similar results may not occur when connected text is used as the 
instructional medium and the intent is to teach concepts. The purpose of the pre- 
sent study was to examine the effect of question level, feedback, and the opportunity 
to restudy in the context of learning concepts. 

Watts and Anderson (1971) developed an interesting methodology for examining 
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the effects of question level on concept learning. Subjects read passages which 
expJained psychological concepts and were asked either to answer factual or ap- 
plication adjunct questions while studying. On the posttest. students faced new 
application questions. The Watts and Anderson (1971) materials involved psycholog- 
ical concep.^, as did the Felker and Dapra (1975) study which used similar proce- 
dures and also found a beneficial effect of higher level questions. We eaiployed 
the basic Watts and Anderson (1971) procedure, but used logical concepts in an 
economic context to extend the nature of the subject matter. 

In the present study, students read a short passage explaining fallacies of 
logic and answered either factual or application adjunct questions immediately after 
reading the passage. One day later subjects either were or were not given feedback 
on their test performance and were allowed to reread the passage. Subjects then 
took a test containing items testing new applications of the described fallacies. 
It was hypothesized that subjects who received higher level questions and subjects 
who received feedback would perform better than subjects who did not. In addition, 
we predicted that the combination of higher level question and feedback would be 
superior to either of the variables singly (a question level by feedback condition 
interaction) . 

Method 

Subjects . The subjects were 135 volunteers from introductory psychology 
classes at Iowa State University; extra course credit was allowed for participating. 

Design . A three (Type of question) X three (Type of feedback) ANOVA represents 
the design. Subjects received either Factual or Apfxication or Both types of ad- 
junct questions immediately after reading the instructional passage. One day later 
subjects were given either: No Feedback, the correct answer to the adjunct questions 
(Correct Answer Feedback), or a list of items gotten wrong and right, the instructional 
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9 passage and instructions to find the correct answers to Items gotten wrong (Self- 
Correction Feedback). There were nine groups formed by the combinations of the 
levels of the 2 factors. 

Materials. A 600 word text explaining six common fallacies of logic and 
using economic examples to clarify these concepts constituted the passage. For 
each fallacy, three application and three factual questions were written. Appli- 
cation questions asked students to select an example of the fallacy from among four 
alternatives: factual questions asked students to select the correct verbatim end- 
ing of a text sentence from among 4 alternatives. Six of the factual and applica- 
tion questions were used to construct adjunct questions that were given immediately 
after the pass£.ge. There were three versions of these adjunct questions tests: a 
6 item factual version, a 6 item application version, and a 12 item combined version. 
The adjunct questions were combined with the remaining factual and application items 
to construct a final posttest. From this posttest four scores were derived for each 
subject: performance on repeated factual, repeated application, new factual and 
new application questions. In order to control for question specific effects, the 
total set of 18 factual and 18 application questions were each divided into 3 sub- 
sets of 6 items each. Use of these subsets as adjunct items was counterbalanced 
within each of the nine treatment combinations. 

The procedure for counterbalancing waa tedious but simple. The 18 Factual 
and 18 Application items were each divided into 3 sets of 6 which may be labeled 
sets A, B and C. The A, B, and C sets of Factual and Application items each served 
as adjunct items for about one-third of the subjects in their appropriate conditions. 
For the mixed condition, the A, B and C set of Factual and Application items were 
paired to make 3 sets of 12 items which may be labeled sets AA, BB and CC. Order 
of items within the adjunct items tests was Independently randomized for each set. 
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In scoring the posttest the parallel between the A, B and C sets of Factual and 
Application items was maintained. For example, for subjects who received the 
A set of Factual items as adjunct questions, the A set Application items were 
counted as Repeated Application items and so on. 

Procedure . The study was run in a typical college classroom. Prior to the 
first session of the experiment, booklets coded for each of the nine treatment 
combinations had been unsystematically intermixed. Subjects were assigned to con- 
ditions by distributing the booklets in that random order. In the first session, 
subjects read the instructional passage at their own pcce. When done, subjects 
raised their hands, the experimenter retrieved the passage booklet and gave the 
subjects the appropriate adjunct questions. Subjects were not permitted to review 
the passage to answer the adjunct questions. One day later, subjects returned and 
were given feedback appropriate to their condition. Subjects in the Self-Correction 
conditions were told which items they got wrong and reread the passages to discover 
correct answers to those items. Tnese subjects indicated their choice of new cor- 
rect answer while doing so. Subjects in the Correct Answer conditions were given 
the correct answers to all items and were told to reread the passage and pay close 
attention to material related to items gotten wrong. Subjects in the No Feedback 
conditions were given no information about test performance and were simply given the 
passage to reread. Subjects had as long as they wished to complete the rereading; 
when done, all subjects took the 36 item post^ st. 

Results 

Table 1 contains the mean arcsine transformed percent correct on the adjunct 
questions and the mean number of items correct for each of the four types of posttest 
scores. Separate 3X3 analyses of variance were computed for each of these five 
types of scores. Since the number of subjects differed between conditions, an un- 
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weighted mean solution was used. The major purpose of the study was to determine 
the effects of Type of Question and Types of Feedback on learning concepts, there- 
fore, performance on the new application questions represents the variable of main 
interest. Performance on each of the variables is discussed in turn. 

Adjunct Items . Since there were different numbers of items for the mixed, 
factual and application groups, the arcsine transformed percent correct served as 
the measure of performance. Only the effect of Type of Question proved significant. 
F (2, 126) = 3.513, £ < .032, MS error = .0836. The means were: Factual = 1.088, 
Application = 1.098, Both = 0.952. Basically, the factual and application groups 
outperformed the groups receiving both types of items. 

New Application . Only the effect of Type of Question proved significant, 
F (2, 126) = 3.858. £ < .024, MS error = 5.164. The means were: Factual = 8.438, 
Application = 9,512. Both = 8.280. Both the Application - Factual and Application - 
Both group differences were significant when tested by the Newman-Keuls procedure. 
Basically, the groups receiving only adjunct application questions did better on new 
applications of the presented concepts than did the other groups. 

New Factual . None of the differences between any of the conditions proved 
significant for t.ie New Factual items. TYe means for each condition are given in 
Table 1. 

Repeated Application . Only the main effect of Type of Question provad signifi- 
cant. F (2. 126) = 4.384. £ ^ .0144. MS error =1.336. The means were: Factual = 
4.268. Application = 4.993. Both = 4.678. Only the difference between the Applica- 
tion and Factual groups proved significant when tested by the Newman-Keuls procedure. 

Repeated Factual . Only the main effect of Type of Question proved significant, 
F (2. 126) = 5.551. £ < .00489. MS error = 1.378. The means were: Factual = 4.762, 
Application = 4.202. Both = 5.014. The Factual and Both groups significantly out- 
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performed the Application group when tested by the Newman-Keuls procedure. 

Type of Feedback did not prove significant in any analysis. However, the 
way In which the analyses were performed may have masked any effect of feedback. 
It may be that feedback has an effect only on the items for which it is given. 
In the analyses described above, it was never the case that all subjects were 
being tested on items that they had received previously. For example, in the anal- 
ysis of repeated factual questions, only subjects in the Factual and Both conditions 
had received those questions previously; subjects in the Application condi ions 
had not seen those items before. The fact that the items were truly repeated for 
some subjects and not for others may have masked any effect of feedback. 

In addition, Phye (Note 2) had recommended that an analysis of the types of 
post test errors be made when feedback is studied. Ph} lassified errors into 
repeated, different, and new errors. Repeated errors are choices of the same wrong 
alternatives originally chosen, different errors are choices of a different wrong 
alternative from the one chosen previously, and new errors are errors on 
items initially correct. Phye has argued that the analysis of such errors may 
reveal effects not found in an overall analysis. 

For these two reasons, a 3 (Type of Question) X 3 (Type of Feedback) X 3 (Type 
of Error) unweighted means supplementary analysis was performed on the repeated 
items. Type of Error was a within subject factor. The dependent measure in this 
supplementary analysis consisted of the arcsine transformed percent of repeated, 
different, and new errors for subjects in each of the conditions. The denominator 
used in computing the percents was the number of adjunct items. Thus, for subjects 
in the Factual conditions, the percents of repeated, different, and new errors on 
Repeated Factual items were computed. For subjects in the Application conditions, 
the percents of repeated, different, and new errors on Repeated Application items 
were computed; for subjects in the Both conditions, repeated, different, and new 
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errors on both Repeated Factual and Application items were counted. The aii=ilysiG 
revealed significant main effects for Type of Question and Type of Feedback, Fs 
(2, 126) = 6.423 and 3.353, £s < .002 and .038, respectively, MS error = 0.0152. 
The means were: Factual = 35.3, Application = 33.9, Both = 30.0; and No Feedback = 
34.7, Correct Answer = 33.6, Self Correct - 30.9. Basically, the Both conditions 
did better than did the Factual or Application groups and the Self Correct condi- 
tions did better than the Correct Answer or No Feedback conditions. 

Discussion 

The major finding of this study was the beneficial effect of application ques- 
tions on later application. Subjects given Application adjunct questions did better 
than subjects given Factual application questions. Moreover, the improvement oc- 
curred without any loss in learning incidental factual information as performance 
on New Factual items was not different. This finding confirmed the Watts and Ander- 
son (1971) and Felker and Dapra (1975) results and extended those to a nev subject 
matter. This finding apparently supports the educators belief that higher level 
questions facilitate higher level learning. However, the beneficial effects of ad- 
junct application questions were obtained only when they were used in isolation; 
when combined with adjunct factual questions, poor performance occurred on new appli- 
cation Items. 

An explanation for the difference in the effect of tdjunct questions when used 
singly and when used in combination with the factual questions was not immediately 
apparent. Some sort of interference occurs between the factual and application 
items. For example, the processing capacity devoted to answering the factual questions 
may reduce processing capacity available for doing whatever processing is necessary 
for committing the concepts to long term memory. The fact that performance on the 
adjunct items in the Both conditions was lower than in the Factual or Application 
conditions is consistent with this hypothesis. This hypothesis could be tested by 
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varying the position of the items* If the items were inserted in the passage 
rather than massed at the end, then the processing demands of the task should be 
lessened, and subjects in the Both conditions should do as well as subjects in 
the application condition. Whatever the explanation for the difference, it is 
clear that the conditions under which higher level questions will facilitate per- 
formance in practical educational situations are not well understood. Clearly, 
further research is necessary to delineate those conditions. 

Part of the motivation for the present research was to determine if feedback 
would interact with level of question. Specifically, it was predicted that appli- 
cation questions would have greater benefit when -ombined with feedback. This pre- 
diction was not supported. Feedback did not influence performance on either new 
applications or new factual questions* 

Feedback did have an effect on performance on repeated items. The effect was 
revealed in the supplementary analysis of repeated, different , and new errors. Sub-- 
jects given feedback did better on the repeated items and the effect of feedback 
was greater if subjects corrected their previous answers. Importantly, these find- 
ings mean that the effect of feedback was specific to the question asked; subjects 
given feedback better learned to answer those specific questions. However, in 
the case of the application items, feedback did not help the subject learn the 
underlying conrept even if the subject was forced to reanswer and reanswered cor- 
rectly as most subjects did. If feedback had helped in the learning of the concept, 
then subjects given feedback should have done better on thii New Application items, 
they did not. 

The reason why feedback had only a question-specific effect may have to do 
with the nature of the learning task. In the Andre and Sola (1976) study, where 
feedback appeared to influence higher level learning, the materials consisted of 
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simple sentences and the higher invel questions involved simple paraphrases c T 
the sentences. With such materials, it seems likely that subjects who made an 
error on an initial item could use the feedback and restudy to properly re-encode 
the item. Understanding paraphrases is a highly overlearned task for most adults. 

In the present study, the New Application questions involved recognizing new 
examples of presented fallacies, a much more complex task. It may have been the 
case that subjects who made an error had no idea of how to correct that original 
error even given the correct answer and the opportunity to restudy. Tlie students 
may have simply been unable to take the presented verbal information and translate 
it into a procedure for recognizing new examples of the concepts. Larkin and Rief 
(1976) present evidence that many college students can lack such critical reading 
skills. This analysis suggests that, with complex cognitive tasks, feedback and 
restudv will not have a beneficial effect on learning unless the subject possesses 
prerequisite skills that allow use of the feedback. 
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Table 1 

Mean Performance on the Adjunct Questions and on the Four Types of 
Posttest Scores for Each of the Conditions 



TYPE OF SCORE 
Adjunct Items ^ Posttest Items 

Arcsince Transformed Repeated New 

Condition Percent Correct Factual Application Factual Application 



Factual-No Feedback 




1.118 


4.500 


4.429 


9.071 


8.286 


Factual-Correct Answer 




1.024 


4.786 


4.000 


8.429 


8.214 


Factual-Self Correct 




1.112 


5.000 


4.375 


9.062 


8,812 


Application-No Feedback 




1.150 


4.500 


4.786 


9.142 


9.857 


Application-Correct Answer 




1.100 


4.250 


5.050 


8.450 


9.250 


Application-Self Correct 


* 


1.044 


3.857 


5.143 


7.929 


9.423 


Both-No Feedback 




0.984 


4.846 


4.461 


7.923 


7.384 


Both-Correct Answer 




0.909 


3.625 


4.500 


7.687 


8.312 


Both-Self Correct 




0.964 


5.571 


5.071 


8.214 


9.143 
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^Since the Mixed groups received 12 and the Factual and Application groups received 6 adjunct items, performance 
was represented as tht percent correct. The percents were subjected to the arcsine transformation for the anal- 
ysis of variance. For posttest scores the number of items for each type v *s the same across conditions, so the 
number of items correct are reported. For the Repeated questions the maximum score is 6, for the New Items the 
maximum is 12. 



